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A MESSAGE FROM THE FRIEDMAN FOUNDATION: 



OUR CHALLENGE TO YOU 



Our research adheres to the highest standards of scientific rigor. We 
know that one reason the school choice movement has achieved such 
great success is because the empirical evidence really does show that 
school choice works. More and more people are dropping their oppo- 
sition to school choice as they become familiar with the large body 
of high-quality scientific studies that supports it. Having racked up a 
steady record of success through good science, why would we sabotage 
our credibility with junk science? 

This is our answer to those who say we can't produce credible research 
because we aren't neutral about school choice. Some people think that 
good science can only be produced by researchers who have no opin- 
ions about the things they study. Like robots, these neutral researchers 
are supposed to carry out their analyses without actually thinking or 
caring about the subjects they study. 

But what's the point of doing science in the first place if we're never al- 
lowed to come to any conclusions? Why would we want to stay neutral 
when some policies are solidly proven to work, and others are proven 
to fail? 

That's why it's foolish to dismiss all the studies showing that school 
choice works on grounds that they were conducted by researchers who 
think that school choice works. If we take that approach, we would 
have to dismiss all the studies showing that smoking causes cancer, 
because all of them were conducted by researchers who think that 
smoking causes cancer. We would end up rejecting all science across 
the board. 

The sensible approach is to accept studies that follow sound scientific 
methods, and reject those that don't. Science produces reliable empiri- 
cal information, not because scientists are devoid of opinions and mo- 
tives, but because the rigorous procedural rules of science prevent the 
researchers' opinions and motives from determining their results. If 
research adheres to scientific standards, its results can be relied upon 
no matter who conducted it. If not, then the biases of the researcher 
do become relevant, because lack of scientific rigor opens the door for 
those biases to affect the results. 

So if you're skeptical about our research on school choice, this is our 
challenge to you: prove us wrong. Judge our work by scientific stan- 
dards and see how it measures up. If you can find anything in our work 
that doesn't follow sound empirical methods, by all means say so. We 
welcome any and all scientific critique of our work. But if you can't find 
anything scientifically wrong with it, don't complain that our findings 
can't be true just because we're not neutral. That may make a good 
sound bite, but what lurks behind it is a flat rejection of science. 
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Executive Summary 



The impact of Florida’s “A + ” accountability program, which until 2006 included a voucher program for chronically 
failing schools, on public school performance has been extensively studied. The results have consistently shown a positive 
effect on academic outcomes in Florida public schools. However, no empirical research has been done on the impact 
of the program after 2002-03, the first year in which a substantial number of students were eligible for vouchers. This 
narrow focus has left the public with an incomplete picture of the true role of vouchers in producing the public school 
improvements associated with the A+ program. 

This empirical study is the first to analyze the effects of vouchers in the A+ program after 2002-03. It includes a 
separate analysis for each year from 2001-02 to 2006-07. Thus, it is the first study to analyze how the program’s effects 
were changed by the court-ordered removal of vouchers in January 2006. Moreover, it uses superior-quality data that track 
the progress of individual students over time, which have not been available to the general public until recently. 

The study finds that vouchers were a key element driving improvements in public schools from the A+ program. 
At every step, the academic performance of failing public schools in Florida responded to changes in the status of 
vouchers in the A + program. While Florida is still experiencing improvements in student performance, the removal of 
vouchers from the program reduced the magnitude of these improvements. 

Key findings include: 

■ Before vouchers were widely available, the A+ program produced modest improvements in failing public schools. 
In 2001-02, public schools that were threatened with vouchers if they did not improve outperformed other Florida 
public schools by 13 points in math on the state’s developmental scale. 

■ In the first year when substantial numbers of vouchers were available, the program produced dramatically 
larger improvements in failing public schools. In 2002-03, public schools whose students were offered vouchers 
outperformed other Florida public schools by 69 points. 

■ In later years, as voucher participation rates dropped due to procedural obstacles that prevented many eligible 
families from using vouchers, the positive effect was not so large. From 2003-04 through 2005-06, public schools 
whose students were offered vouchers outperformed other Florida public schools by between 20 and 27 points. 

■ The removal of vouchers caused the positive impact on public schools to drop well below what it had been even 
in 2001-02, before vouchers were widely available. Schools that would have been eligible for vouchers in 2006-07 
outperformed other Florida public schools by 11 points. The statistical certainty of this result is more moderate 
than the certainty of the results for earlier years. And schools that would have been threatened with vouchers 
made no gains relative to other Florida public schools (compared to the 13 points they gained in 2001-02). 
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long-standing concern about the serious deficiencies in America's elementary and secondary public schools. 
The best way to improve the quality of education, they believe, is to enable all parents with the freedom to 
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Lost Opportunity: An Empirical Analysis of How Vouchers Affected Florida Public Schools 



Introduction 

Florida’s A+ Accountability Program has become a focal point for empirical research on the impact of school 
vouchers on public schools. This research has shown that the A+ program is responsible for large academic improvements 
in chronically failing Florida public schools, and that the gains are largely attributable to vouchers. However, no research 
has examined the impact of the program after 2002-03, the first year in which a substantial number of students were eligible 
for vouchers. This study, by analyzing results in every year from 2001-02 to 2006-07, provides an unprecedented opportunity 
to examine the relationship between vouchers and public school performance under a variety of conditions. 

Voucher Competition and Public Schools 

Perhaps the most important concern about school vouchers is the effect they have on public schools. Many people 
acknowledge that vouchers help the students who use them, but are worried that they will make public schools worse 
by draining money or by “creaming” the best students. 

However, the evidence on the real-world effect of voucher programs shows that this is not the case. No empirical 
study anywhere in the United States has ever found that public schools had worse outcomes when exposed to vouchers. 
And there is a strong body of empirical evidence showing that vouchers make public schools better, not worse. 1 The 
fears that public schools would be harmed by vouchers have simply failed to materialize. The research consistently has 
found that, where students can use school choice to attend any school, public or private, the public schools make bigger 
academic improvements. 

These findings are counter-intuitive to many people, but they are not hard to explain. One reason vouchers improve 
public schools is because they allow parents to find the right particular school for each individual child. Every child is 
unique and has unique educational needs. Another reason is that vouchers do not actually have an adverse impact on 
school budgets, as fiscal studies have shown. 2 

But probably the most important reason school vouchers improve public schools is because they put parents in 
charge. Vouchers give parents a meaningful way to hold schools accountable for teaching their students. 

Under the current system, it is difficult for families to leave if a school isn’t doing a good job. In other service 
areas, from grocery stores to health care, if a service provider isn’t getting the job done people can switch to another 
provider simply by making the decision to do so. With schools, however, the only way to change is to move - an extremely 
cumbersome and disruptive step. 

Thus, to a large degree schools can take students for granted. They lack the positive incentive for better performance 
that, say, hospitals have because they know they must do a good job or lose patients. This is especially true for schools 
that serve low-income and disadvantaged students. These populations are less able to move in order to find a better 
school. 

With school vouchers, if a public school is providing adequate services, parents can leave their children there and 
be no worse off. But if a public school is not doing an adequate job, parents can go find a private school that will serve 
their children better. And whichever school a family chooses, parents will have the power to hold that school accountable 
for teaching their children, because they have the power to leave if they aren’t being served. 

Where parents are empowered with school choice, schools that don’t adequately teach their students will lose them. 
This provides a competitive incentive for better performance. 
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Florida's A+ Accountability Program 

Under the A + Accountability Program, Florida students are tested in reading and math in grades 3 through 10. 
Writing and science tests also are given in some grades. The test results form the basis of a variety of accountability 
measures. For example, students must pass the exam, known as Florida’s Comprehensive Assessment Test (FCAT), to 
be promoted from grade 3 to grade 4 and to graduate from high school. 

Florida gives each public school an annual grade based on its academic performance, primarily measured by the 
results of the FCAT. Until 2006, if a school got two failing grades in any four-year period, its students could apply for 
school vouchers that would pay for their tuition at a private school of their choice, up to a fixed dollar limit. 

In January 2006, the Florida Supreme Court ruled that school vouchers could no longer be offered under the A + 
program. The rest of the accountability program remained in place, but the voucher component was shut down. 



REACTION TO THE fLORim SOPRRV^ OOUKT DECISION STRMTJ& OOAJ^j A*. \^CU(^ERS 

In Bush v. Holmes, the Florida Supreme Court ruled that the Opportunity Scholarship Program, the school voucher 
component of the A + program, violated the state constitution’s requirement that the state provide a “uniform” system 
of education. Wisconsin and Ohio courts had both previously rejected nearly identical challenges to their voucher 
programs. Commentators were quick to point out that the Florida decision contained numerous false assumptions, 
internal inconsistencies, factually inaccurate assertions and logical fallacies. 

In a barely coherent 5-2 decision, Florida’s Supreme Court used recklessly broad language to overturn the 
state’s private school voucher program. . . . Regardless of what one thinks about school vouchers, this 
decision is a serious step backwards for school improvement. More than anything else, it is an educational 
bridge back to the 20th century. While the majority in the court decision seems not to have thought of it, 
any effort to honestly or faithfully apply their decision spells a death sentence for a number of popular 
reforms in Florida and sets an unfortunate precedent for elsewhere. Public charter schooling? Certainly not 
uniform in provision or operations. Specialty schools and tutoring programs? Neither uniformly available 
nor operationally standard. Programs for gifted students? Hardly uniform. Efforts by some districts to 
promote collaborative management, reward outstanding teachers, or adopt new curricular models? None 
of this is uniform either. 

—Frederick Hess and Andrew Rotherham 
Tech Central Station, Jan. 24, 2006 

Despite stating that public schools were the “sole means” for providing education with public funds and 
that they “make no distinction between a small violation of the constitution and a large one,” the majority 
ruled that private [school] placements of students with disabilities was just fine under the constitution. 

Why? Perhaps because ending this practice would have angered a larger political constituency than the few 
hundred largely minority and poor students using the Opportunity Scholarship vouchers? . . . The majority 
found that the voucher program violated the requirement that schools be “uniform.” They observed that . . 

. private schools are “not ‘uniform’ when compared with each other or the public system”. . . If “uniform” 
means all schools must be governed by identical rules for operating procedures and pedagogical techniques, 
then school districts themselves, with their various boards and procedures, are not uniform and should 
also be held in violation. But if “uniform” means that all schools are expected to help students meet the 
same state standards for academic achievement, then the private schools in the voucher program satisfy 
that definition. 

—Jay Greene and Marcus Winters 
Philadelphia Inquirer, Jan. 13, 2006 
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The last feature of Bush v. Holmes that is so distressing is its ready embrace of the story that the use of 
voucher programs necessary diverts needed resources from the public school system. That view of the 
world is hopelessly static, especially in connection with a constitutional provision that actually cares about 
efficiency and high quality education. Viewed dynamically, the removal of children from public schools has 
at least two effects above and beyond the simple diversion of resources. The first of these is that it reduces 
the [financial] obligations of the public school systems, especially when the per-pupil cost of education 
within the state system is higher than the cost of education within the public system, as I suspect it is in 
Florida. What is so horrible about a higher level of funds on a per-capita basis for the students left behind? 
In addition, the private school options, secular or religious, inject a measure of competition. The public 
school teachers and their unions now realize that they are in competition with . . . versatile institutions 
that they cannot control with the drop of a hat. The only way they can maintain their market share is to 
provide, as the Florida Constitution requires, a high quality education in an efficient fashion. 

—Richard Epstein 
University of Chicago Law School Faculty Blog, Jan. 16, 2006 

The court . . . argued that the word “uniform” means that the state must utilize only public schools in 
providing “high quality education.” This even though many public schools are providing nothing of the sort; 
the public school that Octavia [Lopez] would have to attend were she not at Archbishop Curley has been 
rated a failing school for three consecutive years by the state. And even though the state can continue to 
utilize private schools for educating some disabled students. And even though, by the court’s reasoning, 
it is unconstitutional for the state to use the OSP [vouchers] to help Octavia receive a fine education at 
Archbishop Curley, the constitutional mandate about “high quality education” requires consigning her to 
a failing school. And even though there is no evidence that the drafters of the constitution’s language or 
the public that ratified it thought it meant what the Supreme Court now says it means - that in providing 
quality education, the state must enforce a public school monopoly on state funds. 

—George Will 

Washington Post, March 23, 2006 

The Florida Supreme Court struck down one of Florida’s three choice programs on the grounds that it 
created competition for public schools— the very competition that research shows helps drive improvement 
in Florida’s schools. . . . Taking away an alternative for students and parents to choose, there is no incentive 
for schools to make improvements, and it is the children, the future of our state, who suffer. 

—Governor Jeb Bush 

School Choice Advocate, August 2006 



Obstacles to Voucher Participation in the A+ Program 

There were a number of unusual obstacles, not present in most voucher programs, that kept participation rates 
for A+ vouchers unusually low. A certain amount of this kind of difficulty is inherent in the “failing schools” model of 
vouchers, where students’ eligibility for vouchers is determined by the academic performance of their public schools. 
Every year, a new set of parents need to be reached with the message of what vouchers are and that they are eligible 
for them; there can be no gradual building up of information about the program among a fixed population that is always 
eligible. However, in the A+ program the worst obstacles were not inherent in the program design, but were imposed 
by the Florida Department of Education as part of its implementation. 

If a school got an F, and also had at least one F in any of the three previous years, students who were enrolled in 
that school the previous spring could apply for a voucher. They could begin using the voucher the following fall. Students 
were eligible only if they were enrolled in the school in the spring of the year in which the second (or subsequent) 
F was assigned. Students enrolled at that school in future years were not eligible, unless of course the school got yet 
another F. 
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School grades were (and still are) announced in the summer. The timing of the announcement does not follow a 
set schedule every year and thus is unpredictable. Once the state announced school grades, determining which schools 
were eligible for vouchers, parents had only two weeks to apply for the program. If they missed this fleeting application 
window, they could not apply later. 

The extremely short two- week eligibility window was a major obstacle to participation. This difficulty was greatly 
compounded because parents did not know whether they were eligible until school grades came out, at which point 
the two-week clock began ticking. Moreover, parents did not even know when the announcement of the grades might 
be coming. 

This combination of factors made it extremely difficult to keep parents informed. Nobody knew in advance of 
the application window which parents would be eligible, and nobody knew in advance when the announcement would 
come. And once the announcement came, there were only two weeks to get from a starting point of zero information 
to an ending point of parents submitting their applications for vouchers. Many— possibly even most— parents probably 
would not even have known that their children were eligible for the program until it was too late to apply. 

Another disadvantage is the dynamic of media coverage, which is where most parents get their information about 
programs of this type, especially when there is no opportunity for a long-term building of knowledge in a fixed population 
by word of mouth. When two schools were designated as eligible for vouchers in 1999-2000, naturally media attention 
focused on those two schools. This would have helped parents in those neighborhoods find out about the program. Or 
again, in 2002-03, when a substantial number of schools became eligible for vouchers after two straight years with no 
schools eligible, that was a big news story. 

These two years are both instances of how a failing schools voucher program can create “shock value,” but thing 
about shock value is that it doesn’t stay shocking forever. In years after 2002-03, A+ vouchers were no longer the news 
story they were in that year. This would have made it less likely that parents would hear that they were eligible for 
vouchers. 

Another factor was the presence of the court challenge to the program. Uncertainty over a program’s future is a 
barrier to participation. Where there is a reasonable chance that a program might be cancelled at the whim of a judge, 
possibly even in the middle of a school year, parents likely will take into consideration the disruption that might cause 
their children and the problems they might have dealing with their local schools after being forced to return there. 

The Florida lawsuit was probably not a major factor in earlier years, because Florida has a constitutional provision 
that allows government programs to continue to operate after being struck down by lower courts if the state is appealing 
to a higher court. This sensible rule prevents programs from being haphazardly eliminated and reincarnated as dueling 
judges play tug-of-war over public policy. Thus there was no actual prospect for the program to be shut down until it 
reached the Florida Supreme Court in the summer of 2005. 

These obstacles help explain why, as a previous analysis has found, the A+ voucher program is the only voucher 
program ever to see long-term declines in its participation rate. 3 Figure 1 provides the number of voucher participants 
and an estimate of the participation rate in each year of the program. The figure shows that even as the number of 
students using vouchers went up, the participation rate mostly went down. In other words, the number of eligible 
students expanded much more rapidly than the number of students actually using vouchers. 4 

Since the A+ program is the only voucher program ever to see a long-term decline in participation rates, it is reasonable 
to attribute this decline to the unique obstacles to participation that were imposed on the program by the Florida Department 
of Education. 
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Previous Research on the A+ Program 

The effects of the A+ program on public schools have been well studied. In several cases, the same researchers have 
conducted multiple studies over time, using similar methods but refining them to improve the scientific quality of the analysis. 
So far there are a total of four studies or study “lines” examining the impact of the A+ program on Florida public schools. 

The first of these studies was published in 2001 by Jay Greene, then of Harvard University and the Manhattan Institute 
(Greene has subsequently joined the faculty of the University of Arkansas as head of the Department of Education Reform 
in its education school; he retains his Manhattan Institute affiliation). At that point there were only two schools that had ever 
been eligible for vouchers, too few to provide a basis for analysis, but the impact of the threat of vouchers on schools that 
might become eligible for them could be studied. Greene found that schools that had received one F grade, and were thus 
threatened with vouchers if they received another one, made gains relative to other Florida public schools. 5 

In subsequent studies, along with Marcus Winters of the Manhattan Institute, Greene divided schools into four categories; 
see the Method section below for details on these classifications. Greene and Winters found that the positive impact of the A+ 
program closely tracked the schools’ distance from vouchers— Voucher Eligible schools made the biggest academic gains, 
followed by smaller gains in Voucher Threatened schools, followed by two categories of schools that had received Ds but no 
Fs. These later studies looked at the one-year gain in test scores made in 2002-03. 

Greene and Winters also addressed the claims that the improvements by failing schools could be attributed to a “stigma 
effect” or to “regression to the mean.” The stigma hypothesis is that F schools improve in order to remove the stigma of being 
labeled as failing, rather than responding to voucher competition. Greene and Winters examined schools that had received 
an F five years previously, but no Fs in the preceding four years. These schools had once been threatened with vouchers if 
they got another F, but now were no longer so directly threatened. Greene and Winters found that, far from seeing gains from 
the stigma effect, these schools actually backslid a little, relinquishing some of the gains associated with being a Voucher 
Threatened school. Meanwhile, the regression to the mean hypothesis is that failing schools are more likely to improve simply 
because they can’t much worse— as a variable approaches zero it becomes more likely to go up rather than down, because 
there’s less room to go down. However, Greene and Winters compared F schools to other schools that had very similar test 
scores but had not received Fs because the school grades are not exclusively based on test scores. They found that the F 
schools made greater gains. 6 

When Greene and Winters’ most recent analysis was published in the journal Education Next in the summer of 2004, 
it was accompanied by an analysis conducted by Rajashri Chakrabarti, then of Cornell University and now an economist 
at the Federal Reserve Bank of New York. Chakrabarti addressed the stigma hypothesis by comparing the improvements 
made by F schools under the A+ program with improvements made by schools in the lowest performance category (out of 
four) under the state’s earlier accountability program, which had no voucher component. She found that under the earlier 
program, putting a school in the lowest-performing category did not improve its performance relative to schools in the next- 
lowest category, while F schools did outperform D schools under the A+ program. Like Greene and Winters’ studies, this 
analysis only examined scores through 2002-03. 7 

Chakrabarti has subsequently conducted another analysis of the A+ program, comparing its impact on public schools 
to that of the Milwaukee voucher program. In the most recent version of the study, first circulated in 2006, she found that the 
A+ program had a larger impact on public schools. She proposed that this might be attributable to the A+ program being 
a failing schools program, which applies “shock value” by concentrating the exposure to competitive incentives on schools 
most in need of them. 8 This analysis also examines scores only through 2002-03. 9 

David Figlio of the University of Florida and Cecelia Rouse of Princeton University have also studied the A+ program to 
examine its impact on public schools. In their initial analysis, circulated in 2004, they examined data up through 2000-01 (that is, 
before vouchers became widely available in 2002-03) and found that the A+ program improved public schools. By comparing 
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results under the A+ program and the state’s previous accountability program, they find that some of the improvement was 
attributable to competitive incentives from vouchers, but not a majority of the effect . 10 

In a subsequent study released in 2007, in which they are joined by Jane Hannaway of the Urban Institute and Dan 
Goldhaber of the University of Washington, they do not differentiate between the impact of vouchers and other factors associated 
with getting an F grade. They still find that under the A+ program the F grade produced significant improvements in public 
schools. This study collected data through 2004-05, but it used these data only to track the continuing effects on schools that 
had received Fs in 2002-03. It found that the academic improvements brought about by receiving an F grade in 2002-03 were 
sustained in future years. However, this does not provide any information on whether the impact of getting an F grade in 
2002-03 was different than the impact of getting an F grade in other years. 11 

Finally, in 2005 Martin West and Paul Peterson of Harvard University released an analysis that found the A+ program 
produced academic gains in failing schools. This study did not differentiate between the impact of vouchers and the impact 
of other aspects of the program, such as the stigma effect. It examined the impact of the A+ program in 2002-03. 12 

Method 

This study examines the impact of A+ vouchers on Florida public schools in each year from 2001-02 through 2006-07. Since 
no previous study has examined the impact of the program after 2002-03, this study provides the first opportunity to examine 
in greater detail the long-term role of vouchers in improving public schools through a school accountability program. 

The analysis uses Greene and Winters’ school classification system. Schools are designated as Voucher Eligible, Voucher 
Threatened, Always D or Ever D, as follows: 

■ Voucher Eligible schools received two or more Fs in the four preceding school years. These schools are already exposed 
to competitive incentives from vouchers. 

■ Voucher Threatened schools received exactly one F in the preceding four years. These schools will be made eligible for 
vouchers if they receive another F. 

■ Always D schools received only Ds in the preceding four years. These schools are not yet directly threatened with vouchers, 
but there is a serious prospect that they will become so. 

■ Ever D schools received at least one D but no Fs in the preceding four years. These schools are not close to the voucher 
threat, but they are nonetheless towards the lower end of the accountability system. (Always D schools are included in 
both the Always D and Ever D categories.) 

This study uses developmental scale scores that track the academic progress of students over time on a single scale 
that stretches from grades 3 to 10. Florida has recently made available separate test score results, measured in terms of 
year-to-year gains in developmental scale scores, for students in each grade level at each school whose scores are matched 
from one year to the next. Thus, each grade level at each school is a separate case in the analysis. 

Tracking individual students over time is a much better way to measure academic progress, because it removes most 
of the impact of confounding variables such as demographic factors and unobserved characteristics. While the data in this 
study are school-level, they represent the school average for test score results that are individually tracked. 

This study uses math scores on the Sunshine State Standards portion of the FCAT, the portion that is aligned with the 
state’s achievement criteria and is used for the A+ program. It uses math scores because it is statistically easier to discern 
the effects of policy changes with math scores than with reading scores— a phenomenon that is probably due to the greater 



16 



March 2008 




Lost Opportunity: An Empirical Analysis of How Vouchers Affected Florida Public Schools 



influence of confounding variables on reading scores. It uses the Sunshine State Standards portion because developmental 
scale scores tracked from year to year are only available on this portion of the test; previous research has found that results 
on the Sunshine State Standards correlate closely with the other portion of the FCAT, the Norm-Referenced Test portion. 13 

Using these data, a linear regression analysis is performed. The year-to-year gain in developmental scale scores is the 
dependent variable. The independent variables are the percentage of students at each school who are white, the percentage 
of students who are qualified for free and reduced-price lunch, the percentage of students who are Limited English Proficient, 
dummy variables (0 or 1) for whether the school is Voucher Eligible, Voucher Threatened, Always D or Ever D, and dummy 
variables for each grade level. Results are weighted by the number of students in each school. 

All data were obtained from the website of the Florida Department of Education. For the three demographic variables 
and the weights, data from the state’s Adequate Yearly Progress reports are used, because this was the only up-to-date data 
source that included data on the percentage of Limited English Proficient students. 

Adequate Yearly Progress data are not available before 2003-04, so for 2001-02 and 2002-03 the analysis uses data from 
2003-04. The only alternative to this approach would be to go without data on Limited English Proficient students, an especially 
important demographic variable for analyzing Florida schools. 

Files containing school grades are not available before 2002-03. While newer school grade files do include grades from 
older years, they do not include schools that are no longer operating. Since some schools that were eligible for vouchers at 
some point during the period of this analysis were subsequently closed, it is important to use contemporary reports of school 
grades rather than using the latest school grade files, which lack these schools. For 2001-02, however, a contemporary file is 
not available and the analysis has to rely on the file for 2002-03. Any schools that closed in 2001-02, and are therefore not in 
the 2002-03 school grade file, will be excluded from the analysis. 

Results 

The results of the analysis are listed in Table 1. Results for Voucher Eligible and Voucher Threatened schools are 
also presented in Figures 2 and 3. 

The results indicate that vouchers were a key element driving the A+ program’s positive effect on Florida public 
schools. Over multiple years, the impact of the A+ program on the academic performance of failing public schools 
changed, sometimes dramatically, in ways that corresponded to the status of A+ vouchers. Such substantial changes 
should not be occurring if vouchers were not an important contributor to the program’s positive effect. The stigma 
of the F grade, regression to the mean and other factors did not change much from year to year between 2001-02 and 
2006-07, so if they are the only things driving the results we should not see large changes in those results from year to 
year. While the stigma of an F grade or other factors may contribute to the impact of the A+ program, it is difficult to 
avoid the conclusion that competitive incentives due to vouchers were an important influence. 

In 2001-02, before vouchers were widely available, the A+ program produced modest improvements in failing public 
schools. Voucher Threatened schools outperformed other Florida public schools by 13 developmental scale points in 
math on the state’s developmental scale. Always D schools made improvements of 8 points over other Florida public 
schools, while Ever D schools made no improvements relative to other public schools. There were no Voucher Eligible 
schools in this year. 

In 2002-03, the first year when substantial numbers of vouchers were available, the program produced dramatically 
larger improvements in failing public schools. Voucher Eligible schools outperformed other Florida public schools by 
69 developmental scale points. Voucher Threatened schools outpaced other Florida public schools by 15 points, Always 
D schools by 8 points, and Ever D schools by 3 points. 
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From 2003-04 through 2005-06, as voucher participation rates dropped— a change attributable to the procedural 
obstacles discussed above, which would have prevented many eligible families from using vouchers —the positive effect 
was not so large. Voucher Eligible schools outperformed other Florida public schools by between 20 and 27 developmental 
scale points. Voucher Threatened schools improved relative to other schools by 10 points in 2003-04 and 9 points in 2004-05, 
but did not outperform other schools in 2005-06. Always D schools did not outperform other public schools in any year; 
in 2003-04 they actually underperformed other Florida public schools. Ever D schools fluctuated between making no 
improvements relative to other public schools and outpacing them by 6 points. 

In 2006-07, the removal of vouchers caused the positive impact on public schools to drop well below what it had 
been even in 2001-02, before vouchers were widely available. Schools that would have been Voucher Eligible in 2006-07 
outperformed other Florida public schools by 11 points. And schools that would have been Voucher Threatened made no 
gains relative to other Florida public schools (compared to the 13 developmental scale points they gained in 2001-02). 
Always D and Ever D schools also did not outperform other public schools. 

The statistical certainty is of the result for Voucher Eligible schools in 2006-07 is more moderate than the certainty 
of the results for earlier years. The significance value of the result is 0.085, which means, roughly speaking, that we 
can be 91.5 percent certain that the 11-point improvement is attributable to vouchers and not a statistical fluke. The 
conventional significance value required for the full acceptance of a result is 0.05, or 95 percent certainty. However, 
it is an accepted practice to report results with significance between 0.1 and 0.05 (that is, 90 to 95 percent certainty) 
as moderately significant. Thus, we can say that it is likely that the A+ program produced an 11-point gain in schools 
that would have been Voucher Eligible in 2006-07, but we cannot be as certain of this as we can of the other positive 
results. 

Conclusion 

The empirical evidence shows that the loss of vouchers in the A+ program has struck a shocking blow to students in 
Florida’s failing public schools. A+ vouchers were a powerful policy tool for improving the public schools most in need of 
help. This loss only redoubles the urgency of establishing and expanding other school choice programs that will apply the 
healthy incentives that public schools need to improve. 
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Figure 1 



Obstacles to Participation Prevented Many Students 
from Using A+ Vouchers 




Note: In each year, the number of students eligible for vouchers was calculated by taking the highest annual enrollment in a voucher-eligible year at 

each public school that had been eligible for vouchers. 



Figure 2 



Impact of the A+ Program on Voucher Eligible Schools 



80 



+ 69 




2001-02 2002-03 2003-04 2004-05 2005-06 2006-07 



Note: Result in 2006-07 is moderately significant, meaning we cannot be quite as sure that the program produced the observed improvement, rather than a statistical 
fluke. Results in other years are fully significant, meaning we can be certain the program did produce the improvements. 
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Figure 3 
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Table 1 







The Impact of Vouchers on Florida 


Public Schools 






Year 


School Type 


Size of Effect 


Significance 


Number of Schools 


Students Using Vouchers 

Number % of Eligible 


2001-02 


Voucher Eligible 


N/A 


N/A 


0 


47 


5.4% 




Voucher Threatened 


+ 13 developmental scale points 


0.000*** 


75 








Always D 


+ 8 developmental scale points 


0.000*** 


172 








Ever D 


( + 0 developmental scale points) 


0.769 


634 






2002-03 


Voucher Eligible 


+ 69 developmental scale points 


0.000*** 


9 


556 


6.9% 




Voucher Threatened 


+ 15 developmental scale points 


0.000*** 


119 








Always D 


+ 8 developmental scale points 


0.009** 


66 








Ever D 


+ 3 developmental scale points 


0.021* 


622 






2003-04 


Voucher Eligible 


+ 20 developmental scale points 


0.014* 


6 


640 


3.8% 




Voucher Threatened 


+ 10 developmental scale points 


0.008** 


77 








Always D 


-7 developmental scale points 


0.041* 


34 








Ever D 


( + 2 developmental scale points) 


0.190 


457 






2004-05 


Voucher Eligible 


+ 27 developmental scale points 


0.000*** 


18 


763 


2.9% 




Voucher Threatened 


+ 9 developmental scale points 


0.005** 


80 








Always D 


(-2 developmental scale points) 


0.701 


16 








Ever D 


+ 6 developmental scale points 


0.000*** 


405 






2005-06 


Voucher Eligible 


+ 24 developmental scale points 


0.000*** 


24 


740 


2.2% 




Voucher Threatened 


( + 2 developmental scale points) 


0.577 


122 








Always D 


(-3 developmental scale points) 


0.524 


13 








Ever D 


+ 3 developmental scale points 


0.035* 


380 






2006-07 


Voucher Eligible 


+ 11 developmental scale points 


0.097t 


12 


N/A 


N/A 




Voucher Threatened 


(-5 developmental scale points) 


0.110 


90 








Always D 


( + 3 developmental scale points) 


0.495 


16 








Ever D 


( + 1 developmental scale point) 


0.467 


346 







Note: Results are for the average annual change in developmental scale scores in each grade level of each school, for scores individually matched to students across years, on the FCAT-SSS math test. “Voucher Eligible” schools 
are Florida public schools that recieved at least two F grades in the preceding four years; “Voucher Threatened” schools are Florida public schools that recieved exactly one F grade in the preceding four years; “Always D” 
schools are Florida public schools that recieved four D grades in the preceding four years; and “Ever D” schools are Florida public schools that recieved at least one D and no Fs in the preceding four years (including Always D 
schools). The “Significance” column gives the p-value for each result; t = P < = 0.1; * = p <= 0.05; ** = p <= 0.01; *** = p <= 0.001. Control variables were included for the percentage of students in each school who are white, 

low-income, and limited English proficient, as well as dummy variables for each grade level other than 4. 
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